Measuring for Success: Evaluating Leadership Training Programs for Sustainable Impact.

Background: In an era of global health security challenges such as the COVID-19 pandemic, there is greater need for strong leadership. Over the past decades, significant investments have been made in global health leadership development programs by governments and philanthropic organizations to address this need. Evaluating the societal impact of these programs remains challenging, despite consensus on the importance of public health leadership. Objective: This article identifies the gaps and highlights the critical role of monitoring and evaluation approaches in assessing the impact of global health leadership programs. Importantly, we also propose the theory of change (TOC) as a common framework and identify a set of tools and indicators that leadership programs can adapt and use. Methods: We carried out an informal review of major global health leadership programs, including a literature review on leadership program evaluation approaches. Current practices in assessing the short- to long-term outcomes of leadership training programs were explored and synthesized. We also examined use of program theory frameworks, such as theory of change to guide the evaluation strategy. We find the TOC approach can be enhanced by integrating evaluation-specific frameworks and establishing broad stakeholder buy-in. We highlight measurement challenges, proposed outcome indicators and evaluation methodologies, and outline the future direction for such efforts. Findings: Most evaluation of current leadership programs is focused on short-term individual-level outcomes, while reports on long-term societal impact were limited. Reciprocal impacts on and benefits for the "host" organizations were not included in evaluation metrics. Most programs had program logic or result chains, but with no well-articulated program theories. Conclusion: Key stakeholders involved in leadership training programs benefit from the evidence of rigorous program evaluations to inform decisions that address barriers in fostering global health leadership and improving population health outcomes. Insight into reciprocal change in host organizations is important. Evaluation of global health leadership training must go beyond the individual trainee and encompass organizational and community-level impacts. Documentation of long-lasting organizational and societal impacts is essential for donors to appreciate the return on their investment. Key Takeaways: Evaluation plays an important role in understanding how leadership development takes place and how it contributes to improving public health outcomes.Making the case for investments in leadership development programs requires robust evidence from monitoring and evaluation strategies that link investments beyond the individual-level to longer-term societal impacts.The first critical step towards a strategy for success is for leadership programs to clearly build, articulate, share, and use their program theories or theories of change.Theories of change help identify the pathways (and potential tensions) through which leadership development programs effect change at the individual, organizational and community levels.Evaluation methods that examine outcomes of leadership programs should be multi-method, multi-level, and where possible include counterfactual outcomes.Allocation of funds to evaluate on-going and long-lasting societal impact of leadership programs should be a routine practice.

Strengthening Public Health Leadership in Africa: An Innovative Fellowship Program.

PROBLEM: The Ebola virus disease crisis in West Africa revealed critical weaknesses in health policy and systems in the region, including the poor development and retention of policy leaders able to set sound policy to improve health. Innovative models for enhancing the capabilities of emerging leaders while retaining their talent in their countries are vital. APPROACH: Chatham House (London, United Kingdom) established the West African Global Health Leaders Fellowship to help develop the next generation of West African public health leaders. The innovative program took a unique approach: Six weeks of intensive practical leadership and policy training in London and Geneva bookended a 10-month policy project conceived and carried out by each fellow in their home country. The program emphasized practice, site visits and observation of U.K. public health organizations, identifying resources, and networking. Strong mentorship throughout the fieldwork was a central focus. Work on the pilot phase began in June 2016; the fellows completed their program in September 2017. OUTCOMES: The pilot phase of the fellowship was successful, demonstrating that this "sandwich" model for fellowships-whereby participants receive focused leadership training at the start and end of the program, minimally disrupting their lives in-country-offers exciting possibilities for enhancing leadership skills while retaining talent within Africa. NEXT STEPS: On the basis of this successful pilot, a second cohort of eight fellows began the program in October 2018. The expanded African Public Health Leaders Fellowship has become a central activity of Chatham House's Centre on Global Health Security.

Creating a global dialogue on infectious disease surveillance: connecting organizations for regional disease surveillance (CORDS).

Connecting Organizations for Regional Disease Surveillance (CORDS) is an international non-governmental organization focused on information exchange between disease surveillance networks in different areas of the world. By linking regional disease surveillance networks, CORDS builds a trust-based social fabric of experts who share best practices, surveillance tools and strategies, training courses, and innovations. CORDS exemplifies the shifting patterns of international collaboration needed to prevent, detect, and counter all types of biological dangers - not just naturally occurring infectious diseases, but also terrorist threats. Representing a network-of-networks approach, the mission of CORDS is to link regional disease surveillance networks to improve global capacity to respond to infectious diseases. CORDS is an informal governance cooperative with six founding regional disease surveillance networks, with plans to expand; it works in complement and cooperatively with the World Health Organization (WHO), the World Organization for Animal Health (OIE), and the Food and Animal Organization of the United Nations (FAO). As described in detail elsewhere in this special issue of Emerging Health Threats, each regional network is an alliance of a small number of neighboring countries working across national borders to tackle emerging infectious diseases that require unified regional efforts. Here we describe the history, culture and commitment of CORDS; and the novel and necessary role that CORDS serves in the existing international infectious disease surveillance framework.

Priorities for research on meningococcal disease and the impact of serogroup A vaccination in the African meningitis belt.

For over 100 years, large epidemics of meningococcal meningitis have occurred every few years in areas of the African Sahel and sub-Sahel known as the African meningitis belt. Until recently, the main approach to the control of these epidemics has been reactive vaccination with a polysaccharide vaccine after an outbreak has reached a defined threshold and provision of easy access to effective treatment but this approach has not prevented the occurrence of new epidemics. Meningococcal conjugate vaccines, which can prevent meningococcal carriage and thus interrupt transmission, may be more effective than polysaccharide vaccines at preventing epidemics. Because the majority of African epidemics have been caused by serogroup A meningococci, a serogroup A polysaccharide/tetanus toxoid protein conjugate vaccine (PsA-TT) has recently been developed. Results from an initial evaluation of the impact of this vaccine on meningococcal disease and meningococcal carriage in Burkina Faso have been encouraging. To review how the research agenda for meningococcal disease in Africa has been changed by the advent of PsA-TT and to define a new set of research priorities for study of meningococcal infection in Africa, a meeting of 41 scientists was held in Dakar, Senegal on April 24th and 25th 2012. The research recommendations developed during the course of this meeting are presented in this paper. The need for enhanced surveillance for meningitis in defined populations with good diagnostic facilities in African countries at risk of epidemics was identified as the highest priority. This is needed to determine the duration of protection against serogroup A meningococcal disease provided by PsA-TT and to determine the risk of disease and carriage caused by meningococci of other serogroups. Other research areas given high priority included identification and validation of serological correlates of protection against meningococcal disease and carriage, development of improved methods for detecting carriage and epidemiological studies aimed at determining the reasons underlying the peculiar epidemiology of meningococcal disease in the African meningitis belt. Minutes and working papers from the meeting are provided in supplementary tables and some of the presentations made at the meeting are available on the MenAfriCar consortium website (www.menafricar.org) and on the web site of the Centers for Disease Control (www.cdc.gov).

Ten years of international collaboration in biomedical informatics and beyond: the AMAUTA program in Peru.

Well-trained people are urgently needed to tackle global health challenges through information and communication technologies. In this report, AMAUTA, a joint international collaborative training program between the Universidad Peruana Cayetano Heredia and the University of Washington, which has been training Peruvian health professionals in biomedical and health informatics since 1999, is described. Four short-term courses have been organized in Lima, offering training to more than 200 graduate-level students. Long-term training to masters or doctorate level has been undertaken by eight students at the University of Washington. A combination of short-term and long-term strategies was found to be effective for enhancing institutional research and training enterprise. The AMAUTA program promoted the development and institution of informatics research and training capacity in Peru, and has resulted in a group of trained people playing important roles at universities, non-government offices, and the Ministry of Health in Peru. At present, the hub is being extended into Latin American countries, promoting South-to-South collaborations.

Pandemic influenza planning by videoconference.

Collaboration between nations and sectors is crucial to improve regional preparedness against pandemic influenza. In 2008, a Virtual Symposium was organized in the Asia-Pacific region by the Asia-Pacific Economic Cooperation Emerging Infections Network (APEC EINet) to discuss pandemic preparedness. The multipoint videoconference lasted approximately 4.5 hours and was attended by 16 APEC members who shared best practices in public-private partnerships for pandemic influenza preparedness planning. Twelve of the 16 APEC members who participated responded to a post-event survey. The overall experience of the event was rated highly. Partnering public health, technology and business communities to discuss best practices in preparedness using videoconferencing may be an effective way to improve regional preparedness. Utilization of videoconferencing on a routine basis should be considered to improve preparedness among APEC members and enhance its usability during a pandemic.

Developing capacity in health informatics in a resource poor setting: lessons from Peru.

The public sectors of developing countries require strengthened capacity in health informatics. In Peru, where formal university graduate degrees in biomedical and health informatics were lacking until recently, the AMAUTA Global Informatics Research and Training Program has provided research and training for health professionals in the region since 1999. The Fogarty International Center supports the program as a collaborative partnership between Universidad Peruana Cayetano Heredia in Peru and the University of Washington in the United States of America. The program aims to train core professionals in health informatics and to strengthen the health information resource capabilities and accessibility in Peru. The program has achieved considerable success in the development and institutionalization of informatics research and training programs in Peru. Projects supported by this program are leading to the development of sustainable training opportunities for informatics and eight of ten Peruvian fellows trained at the University of Washington are now developing informatics programs and an information infrastructure in Peru. In 2007, Universidad Peruana Cayetano Heredia started offering the first graduate diploma program in biomedical informatics in Peru.

Regional infectious disease surveillance networks and their potential to facilitate the implementation of the international health regulations.

The International Health Regulations (IHR) 2005 present a challenge and opportunity for global surveillance and control of infectious diseases. This article examines the opportunity for regional networks to address this challenge. Two regional infectious disease surveillance networks, established in the Mekong Basin and the Middle East, are presented as case studies. The public-private partnerships in the networks have led to an upgrade in infectious disease surveillance systems in capacity building, purchasing technology equipment, sharing of information, and development of preparedness plans in combating avian influenza. These regional networks have become an appropriate infrastructure for the implementation of the IHR 2005.

Evaluation of a joint Bioinformatics and Medical Informatics international course in Peru.

BACKGROUND: New technologies that emerge at the interface of computational and biomedical science could drive new advances in global health, therefore more training in technology is needed among health care workers. To assess the potential for informatics training using an approach designed to foster interaction at this interface, the University of Washington and the Universidad Peruana Cayetano Heredia developed and assessed a one-week course that included a new Bioinformatics (BIO) track along with an established Medical/Public Health Informatics track (MI) for participants in Peru. METHODS: We assessed the background of the participants, and measured the knowledge gained by track-specific (MI or BIO) 30-minute pre- and post-tests. Participants' attitudes were evaluated both by daily evaluations and by an end-course evaluation. RESULTS: Forty-three participants enrolled in the course - 20 in the MI track and 23 in the BIO track. Of 20 questions, the mean % score for the MI track increased from 49.7 pre-test (standard deviation or SD = 17.0) to 59.7 (SD = 15.2) for the post-test (P = 0.002, n = 18). The BIO track mean score increased from 33.6 pre-test to 51.2 post-test (P < 0.001, n = 21). Most comments (76%) about any aspect of the course were positive. The main perceived strength of the course was the quality of the speakers, and the main perceived weakness was the short duration of the course. Overall, the course acceptability was very good to excellent with a rating of 4.1 (scale 1-5), and the usefulness of the course was rated as very good. Most participants (62.9%) expressed a positive opinion about having had the BIO and MI tracks come together for some of the lectures. CONCLUSION: Pre- and post-test results and the positive evaluations by the participants indicate that this first joint Bioinformatics and Medical/Public Health Informatics (MI and BIO) course was a success.

Training biomedical and health informatics professionals in Peru: towards the development of the first graduate diploma program in the country.

In Peru, there are no formal university graduate programs in biomedical and health informatics, and there is a lack of other health informatics programs. AMAUTA, a joint program between the Universidad Peruana Cayetano Heredia (UPCH) in Lima and the University of Washington (UW) in Seattle has been training Peruvian health professionals since 1999 regarding biomedical and health informatics. In 2007, UPCH is offering the first Graduate Diploma Program in Biomedical Informatics in the country.

Trade related infections: farther, faster, quieter.

Modern global trading traffics large volumes of diverse products rapidly to a broad geographic area of the world. When emergent infections enter this system in traded products their transmission is amplified. With truly novel emergent infections with long incubation periods, such as Human Immunodeficiency Virus (HIV) or variant Creutzfeld Jacob Disease (vCJD), this transmission may silently disseminate infection to far distant populations prior to detection. We describe the chronology of two such "stealth infections," vCJD and HIV, and the production, processing, and distribution changes that coincided with their emergence. The concept of "vector products" is introduced. A brief case study of HIV incursion in Japan is presented in illustration. Careful "multisectoral" analysis of such events can suggest ecologically critical pathways of emergence for further research. Such analyses emphasize the urgency of implementing safety measures when pathogens enter globally traded products.

The global diet: trade and novel infections.

Practices designed to meet the demands of global trade can amplify food safety problems. Ever-increasing pressure to churn out more product and better sides of beef has generated processes that compromise existing safety measures. Among the concerns are intensified food production, use of antimicrobials and hormones as growth promoters, and poor sanitary infrastructure in some food producing countries. Accompanying the innovations designed to serve the diversifying global palate are emerging and re-emerging infectious diseases, or "trade-related infections." The joint efforts of international public health and industry are required to effectively address these growing health challenges.

Design and implementation of Cell-PREVEN: a real-time surveillance system for adverse events using cell phones in Peru.

With more clinical trials involving evaluations of new drugs or vaccines, monitoring for early detection of adverse events is essential. The overall goal of this study was to develop an interactive-computer system using cell phones for real-time collection and transmission of adverse events related to metronidazole administration among female sex workers (FSW) in Peru. We developed an application for cell phones in Spanish, called Cell-PREVEN, based on a system from Voxiva Inc. We used cell phones to enter data collected by interviewers from FSW in three communities. Information was stored in an online database, where it could be immediately accessed worldwide and exported over a secure Internet connection. E-mail and text messages sent to mobile devices alerted key personnel to selected symptoms. This pilot project has demonstrated that it is feasible to develop a public-health-surveillance system based on cell phones to collect data in real-time in Peru (http://www.prevenperu.org).

Trade related infections: farther, faster, quieter

Modern global trading traffics large volumes of diverse products rapidly to a broad geographic area of the world. When emergent infections enter this system in traded products their transmission is amplified. With truly novel emergent infections with long incubation periods, such as Human Immunodeficiency Virus (HIV) or variant Creutzfeld Jacob Disease (vCJD), this transmission may silently disseminate infection to far distant populations prior to detection. We describe the chronology of two such "stealth infections," vCJD and HIV, and the production, processing, and distribution changes that coincided with their emergence. The concept of "vector products" is introduced. A brief case study of HIV incursion in Japan is presented in illustration. Careful "multisectoral" analysis of such events can suggest ecologically critical pathways of emergence for further research. Such analyses emphasize the urgency of implementing safety measures when pathogens enter globally traded products.